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Abstract 

In the present paper, we analyze symmetric mixed states corresponding to the 
so-called concept formation on a sparsely encoded associative memory model 
with — 1 neurons. Three types of mixed states, OR, AND and a majority 
decision mixed state are described as typical examples. Each element of the 
OR mixed state is composed of corresponding memory pattern elements by 
means of the OR-operation. The other two types are similarly defined. By 
analyzing their stabilities through the SCSNA and the computer simulation, 
we found that the storage capacity of the OR mixed state diverges in the 
sparse limit, but that the other states do not diverge. In addition, we found 
that the optimal threshold values, which maximize the storage capacity, for 
the memory pattern and the OR mixed state coincide with each other in the 
spare limit. Thus, we conclude that the OR mixed state is a reasonable repre- 
sentative of the mixed state in the spare limit. Finally, the paper examines the 
relationship between our results and recently reported physiological findings 
regarding face-responsive neurons in the inferior temporal cortex. 

I. INTRODUCTION 

In this paper, we analyze symmetric mixed states (hereafter "mixed state" ) on a sparsely 
encoded associative memory model. When an associative memory model is made to store 
memory patterns as a result of correlation learning, a pattern, which is a nonlinear superpo- 
sition of the stored memory patterns using a neuron output function, automatically becomes 
the equilibrium state of the model. This is called the mixed state. It is not appropriate 
to think that this mixed state is a side-effect and/or that is unnecessary for information 
processing. Amari has discussed a "concept formation" using the stability of mixed state 
IJ. The correlated attractor proposed in [§,|J, which is a model of the Miyashita attractor 
0, is considered to be a (not symmetric) mixed state in a broad sense. Recently, Parga and 
Rolls have used the mixed state as a mechanism of invariant recognition under a coordinate 
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transformation ||. On the other hand, the sparse coding scheme is believed to be used in 
the brain according to some physiological findings || and the theoretical viewpoints |6|-10 



Thus, the properties of the mixed states on the sparsely encoded associative memory model 
should be discussed. This is the main purpose of the present paper. 

sg n (X^=i £i + h) is the mixed state in a model storing memory pattern with the 
output function sgn(-) and threshold value h. Therefore s types of mixed state can be 
composed of s memory patterns by changing the threshold value. sgn(X^=i £f) among the 
s types is regarded as the mixed state in a model with the output function sgn(-) that is 
made to learn memory patterns of a 50% firing rate [IT]. In other words, a "typical" mixed 
state is a state of the majority decision for the firing or non-firing elements among the s 
types of the mixed state. A natural question then arises: what kind of mixed state among 
the s types should be considered typical in the sparse limit of / — > ? (/is the firing rate of 
the memory pattern). In the sparsely encoded model, storage capacity strongly depends on 
the threshold value, so that an appropriate setting of the threshold value makes the storage 
capacity diverge at l/|/log/| in the sparse limit |6|-[T0|. Thus, in this study, "typical" mixed 



state is defined as the mixed state with similar properties to the memory pattern for the 
storage capacity and the threshold value. The purpose of this paper is to discuss which 
mixed state in the sparse limit should be regarded as a typical mixed state. 

First, three out of s types of the mixed state are considered: the OR mixed state, the 
majority decision mixed state, and the AND mixed state. Each element of the OR mixed 
state is taken from the OR operation in the Boolean sense for each element of memory 
patterns composing of the mixed state. The other two types are similarly defined. The 
examination of the stability of each mixed state by the self-consistent signal-to-noise analysis 
(SCSNA) proposed by Shiino and Fukai [|12[] and the computer simulation shows that the 
storage capacity of the OR mixed state diverges as l/|/log/|, but the storage capacities 
of the other two types do not diverge. This can be explained as follows: the overlap of 
the majority decision mixed state with the stored memory pattern and the overlap of the 
AND mixed state with the stored memory pattern converge at in the sparse limit, whereas 
the overlap of the OR mixed state with one converges at 1. Likewise, while the optimum 
threshold value that maximizes the storage capacity of the memory pattern or the OR mixed 
state settles at the same value, the optimum threshold value of the majority decision mixed 
state and the AND mixed state does not agree with the optimum threshold value of the 
memory pattern in the sparse limit. Therefore, the important factors to decide the storage 
capacity and the optimum threshold value in the sparse limit are asymptotes for the overlaps 
of the s types of the mixed state with the memory patterns. Besides three typical mixed 
states, we also evaluate the overlaps of the s types of the mixed states to show that all the 
overlaps of the s — 1 kinds of mixed states except for the OR mixed states converge to in 
the sparse limit. Thus, we conclude that a consideration of the OR mixed state as a typical 
mixed state in the sparse coding is appropriate. 

At the end of this paper, we discuss the recently reported physiological findings of Sugase 
et al. [fnj on the face-responsive neurons in the inferior temporal cortex. They have reported 
that some face-responsive neurons in the IT cortex code rough classification information in 
the first half of the firing and then code detailed classification information in the latter half 
of the firing. Their findings suggest that the OR mixed state emerges in the first half of 
the neuron's dynamics, and then the network state converges in the memory state. We will 
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infer a relationship between the physiological findings and our present results. 



II. MODEL 

Shown below are the equilibrium properties of a recurrent neural network composed of 
N neurons with an output function ©(•), 

Xi = Q(ui), (1) 

N 

JijXj + h, (2) 



u 



where Xi denotes an output of the i-th neuron in the equilibrium state, and Jy denotes 
a synaptic weight from the j-th neuron to the z-th neuron. The threshold value h of the 
neuron is assumed to not depend on the serial number % of a neuron. Its concrete value is 
described later. Each element r/f of the /i-th memory pattern T7 M , which is stored in the 
present model, is independently generated by the probability, 

Probfaf = 1] = 1 - Prob [r/f = 0] = /, (4) 

where E^f] = /, and / stands for the firing rate of memory pattern rf^ . The pattern with a 
small firing rate / is called a sparse pattern, and the use of a sparse pattern for the memory 
pattern is called a sparse encoding. The synaptic weight is decided by the covariance 
learning method, 

i aN 

JlJ = N f (i - 7) £W - - /} ' (5) 

where aN is a number of the stored memory patterns, and a is defined as the loading rate. 

Let us define the mixed state ^( s,fe ) composed of s memory patterns. The i-th element of 
the mixed state ^y( s > fc ) is set to 'V if the number of the firing state ('1') is k or more in the i-th 
elements of the s memory patterns that compose the mixed state. Otherwise, it becomes 
'0'. The s types of the mixed states exist according to this definition because 1 < k < s. 
In particular, among s types of mixed state, mixed state ^y^- 1 ) is considered to be the OR 
mixed state, where its i-th element is given by the OR operation through the i-th elements 
of s memory patterns. Following the definition of the OR mixed state, ~f( s ' sS > corresponds to 
the AND mixed state where its i-th element is obtained by the AND operation. Moreover, 
j( s , s / 2 ) j s the majority decision mixed state that regards the one with more numbers of or 
1 for the i-th elements of s memory patterns. 

The threshold value h in Eq.(Q) is decided as follows. The threshold value h should be 
appropriately chosen in the sparsely encoded model USHlQj. In this paper, the threshold 



value is calculated by using the mean firing rate of the retrieval pattern. The threshold 
value obtained by this method approximately coincides with the optimum threshold value 
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by which the storage capacity is maximized |Ty]. Therefore, we employ a method with a 
constant firing rate as an approximate method of obtaining the optimum threshold value in 
the present paper. Since the mean firing rate of the memory pattern is /, when the memory 
pattern is retrieved, threshold value h is decided from the following equation, 

1 N ( N \ 

/ = ^E e E^ + M- ( 6 ) 

Moreover, the mean firing rate f( s > k > of mixed state replaces / in Eq. (||) to recall the 
mixed state ~f( s ' k \ and threshold value h is decided, 



f (s,k) = E[7 (^)] = j2 s c v f(i - fy- v . (?) 

v=k 



Here, C is a number of the combination. The overlap of the equilibrium state x with the 
/x-th memory pattern rj 11 is defined, 



1 



Nf(i - f) f 



TM - /)*<■ (8) 



i=l 



If the equilibrium state x is completely equal to T7 M , then m M = 1. The overlap M^ s ' k ' of the 
equilibrium state x with the mixed state 'y( s > k > is defined in a manner similar to the Eq. (^|), 

1 N 

M (s ' fe) = - - f (s > k) )x- (9) 

jV/( s ' fc )(l - /( s ' fc )) ^ 

If the equilibrium state £C is *y( s ' k \ then = 1. 



III. RESULTS 
A. SCSNA analysis 

The SCSNA |TJ] and a computer simulation are used to analyze stabilities of three typical 
types of mixed state, the OR mixed state •y^ s ' 1 \ the majority decision mixed state *y( s ' s / 2 \ 
and the AND mixed state ^ s ' s \ The order parameter equations of the SCSNA are derived 
as shown in the Appendix. 

First, the OR mixed state described. Fig. |l] shows the loading rate a dependency 

of overlap M^ 3 ' 1 ) on the OR mixed state ^t 3 ' 1 ) composed of three memory patterns (s = 
3). The three lines indicate the results of the SCSNA and the data points and the error 
bars are the results of the computer simulation for the mean firing rate of the memory 
pattern, / = 0.5,0.2 and 0.1. In the computer simulation, the neuron number is set as 
N > 10, 000, and the simulation is performed 11 times for each parameter. The data point 
shows the median, and both ends of the error bar show the 1/4 deviation and the 3/4 
deviations, respectively. The horizontal axis is the loading rate a, and the vertical axis is 
the overlap M^ 3 ' 1 ) of the OR mixed state -y^ 3,1 ) with the equilibrium state x. The results 
of the SCSNA and the computer simulation correspond well with each other for the loading 
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rate a dependency of overlap M^ 3,1 ) and the storage capacity a>c, which is the loading rate 
when the equilibrium state becomes unstable. The properties of the three types of the mixed 
states are examined using the SCSNA results. This is because the SCSNA results explain 
the computer simulation results fairly well, as shown in Fig. [I]. Fig. |2] shows the mean 
firing rate / dependency of the storage capacity ac on -yfo 1 ) (s = 3, 5, 7) of the OR mixed 
state. The solid line shows the storage capacity of memory pattern (s = 1). The higher 
the mixed number s, the smaller the storage capacity is with the same mean firing rate /. 
However, the lower the mean firing rate /, the larger the storage capacity for all three states 
of s. Then, we examine the asymptotes for / —>■ 0. The asymptotes for the all values of s 
shown in Fig. || are the same l/|/log/| as the case for the memory patterns. In addition, 
the storage capacity of the OR mixed state 'y^' 1 ) with the s memory patterns with firing 
rate / is the same as that of the memory pattern with firing rate sf in the sparse limit. 
The reason is explained in the next section § |111 B . Fig. f| shows that the mean firing rate 
/ dependency of threshold value h c at the loading rate a is set to the storage capacity, i.e., 
a = ac- The figure shows the threshold value of the memory pattern with a solid line for 
comparison. In the limit of / — > 0, threshold value h c converges to the same value without 
depending on the s that includes the memory pattern s = 1. The reason is also explained 
in the next section § |111 Bj . These findings imply that both of the memory patterns 77^ and 
the OR mixed state 'y( s > 1 ) easily coexist as stable in the sparse limit / — >• 0. 

Next, we examine the majority decision mixed state ^( s ' s / 2 ) and the AND mixed state 
^O. s )_ pjg_ m anc [ g s how the storage capacity for the majority decision mixed state 
j( s ' s / 2 \ (s = 3,5,7) and the AND mixed state ~f( s ' s \ (s = 3,5,7), respectively. Fig. |5| 
shows that the storage capacity gradually increases in s = 3 as the mean firing rate / de- 
creases, and the storage capacity decreases again without diverging in the sparse limit. It is 
about a c = 0.065 even at the maximum. If the number s of memory patterns is increased 
to s = 5, 7, the maximum storage capacity is reduced further, and it goes toward in the 
sparse limit. Fig. | shows that the storage capacity a c of the AND mixed state ~f( s ' sS ) de- 
creases as the mean firing rate becomes small in each number s of the mixed state, and it 
goes toward in the sparse limit. These results show that storage capacity ac does not 
diverge in the sparse limit except in the memory pattern and the OR mixed state •y^ s ' 1 \ 
The reason is qualitatively explained in § 111 B . Fig. ^ and |8| show the mean firing rate / 



dependency of threshold value h c at the storage capacity a c for the majority decision mixed 
state *y( s ' s / 2 \ (s = 3,5,7) and the AND mixed state *y( s > s \ (s = 3,5,7). The figure shows 
the threshold value of the memory pattern with a solid line for comparison. The threshold 
values of these types of mixed state do not correspond to the threshold value of the memory 
pattern in the sparse limit, while that of the OR mixed state the same as that of 

the memory pattern. This means that the memory pattern and these types of mixed state 
cannot coexist in the sparse limit as stable. On the other hand, the threshold value of the 
majority decision mixed state -y( s ' s / 2 ) crosses the threshold value of the memory pattern at 
the mean firing rate of / = 0.5. As mentioned chapter §0, this implies that the majority 
decision mixed state ^( s ' s / 2 ) can be considered as the typical mixed state in / = 0.5 [fiT . 
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B. Qualitative evaluation of mixed state by the naive S/N analysis 



In this section, we explain the analytical results of the SCSNA by using the naive S/N 
analysis P,[10||. First, the reason why only the storage capacity of the OR mixed state ^( Si1 ) 



diverges in the sparse limit is discussed. Assuming that output Xj of each neuron is equal to 

is k) 

the mixed state 7) ' , the internal potential U{ is rewritten with the synaptic weight J^- of 
Eq. (!]). Without loss of generality, the mixed state 7 (s > fe ) is composed of rf ^ (1 < /i < s), 
because the synaptic weight Jij in Eq. ([|) is invariant for the replacement of the order of 
/i. The Ui in Eq. (Q) is decomposed into 1 < ji < s and other parts, 



U; 



N 

E + h 



= Y,W-fH s ' k) + h + z, (10) 

1 N 



aN N 

E EW-fM-fri'*- 



Nf(l-f)^ s+lj ^ 

(12) 

The first term of Eq. (PH|) , which is composed of finite number of overlap m^' k \ is a signal 
term to retrieve. The second term is the threshold value, and Ei the third term is a cross-talk 
noise, which prevents the state ^ s ' k "> from being stable. In this case, z\ obeys the normal 
distribution N(0, af^ s ' k ^) in the limit of iV — > 00. Let us evaluate m^' k ^ in the sparse limit. 
However, we examine k > 1 because all elements of the mixed state with k = become 1. 
The overlap m^'^ of the mixed state 'y( s < k ^ with the memory pattern 77^ can be calculated 
by using the expectation value in memory pattern rf generated with the probability in Eq. 



m^ k) = J^TjfM - f)lt k \ (13) 

s-l 

\ s — v — 1 



E.-ic*r(i-/) f 



v=0 



x [9(1 + v - k) - e(v - k)} . (14) 

Here C is a number of the combination. Note that 0(0) = 1 as in Eq. (|3|). In the sparse limit 
of / — > 0, f v in Eq. ([14]) becomes 1 in v = and in v > 1. Consequently, the summation 
for v of Eq. (T3) has to be taken into account only when v = 0. Thus, (1 — /) s ~ 1 '~ 1 — > 1 for 



/ — » 0, and s _iC„ = 1 for u = 0. 0(0 — fc) = as a result of taking k > 1 into consideration. 



Therefore, the Eq. (O) becomes the following in the sparse limit, 



m (*,fc) = (! _ k y ( 15 ) 
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This expression explains that rrv^ ,k > = 1 only when k — 1, and mM' k > = in k > 2 regardless 
of the number of s in the sparse limit. Fig. |D| shows the mean firing rate / dependency of 
the overlap of m$' k > . This figure also shows that only when k = 1, m^'^ — > 1 in the sparse 
limit, and m[f' fc ) — > in > 2. 

If we use Eq. ([T|), the variance af^ s ^ of cross-talk noise ^ in Eq. (0) becomes, 

a/ M = «E» C /(l-/r. (16) 

The mean firing rate f^ s ' 1 ' of the -y^- 1 ) converges on /ta 1 ) — ► s/ in the sparse limit. Therefore, 
in the sparse limit, the variance of cross-talk noise when the memory pattern with the sf 
mean firing rate is recalled, is asf . That is the case of the mixed state composed a memory 
pattern with / mean firing rate is asf. Thus, they are the same as each other in the sparse 
limit. This is the reason why the storage capacity of the OR mixed state ^ <s > 1 ), which is 
composed of s memory patterns with the mean firing rate /, is the same as that of the 
memory pattern with the mean firing rate sf. In the same way, when the mixed state *y( s < k \ 
which is k > 2, is evaluated on the variance of the cross-talk noise in Eq. (0), it becomes 
in the sparse limit. Thus, both m|f ,fc - ) constituting the signal term and variance for the cross- 
talk noise term become in the sparse limit. When the S/N ratio at that time is calculated 
for k > 2, it converges to 0. Therefore, the study showed that the storage capacity did not 
become only in the OR mixed state -y^' 1 ) of k — 1 in the sparse limit. 

Here we use the naive S/N analysis to discuss why the threshold value of only OR mixed 
state -y^' 1 ) converges at the value excluding 0, while the threshold value converges at for 
k > 2 in the sparse limit. Threshold value h is a boundary value where the neuron state X{ 
with the internal potential Ui is assigned to either 1 or such as Xi = Q(ui + h). Thus, if the 
absolute value of the signal term approaches 0, the threshold value, which is the boundary 
value , goes toward 0. This is why the threshold value converges at on the mixed state 
with k > 2 where the absolute value of the signal term goes toward in the sparse limit. 
On the other hand, m^' 1 ' — > 1 on the OR mixed state -y( s ' 1 J, and the overlap m^' 1 ' on the 
memory pattern is always mfi' 1 ' = 1. Therefore, the threshold value that stabilizes the OR 
mixed state 'y( s ' 1 ) and the threshold value that stabilizes the memory pattern coincide with 
each other at h — —0.5 in the naive S/N analysis. The threshold value obtained from the 
SCSNA as shown in Fig. [| becomes he = —0.7, and it differs somewhat from the result 
h = —0.5. However, this is the qualitative reason for the optimum threshold value of the 
OR mixed state that corresponds to the threshold value of the memory pattern in the 
sparse limit / — >• 0. 

IV. SUMMARY AND DISCUSSION 

The various types of symmetric mixed states on the sparsely encoded associative memory 
model composed of — 1 neurons were investigated using the SCSNA and the computer 
simulation. The results showed that the storage capacity of the OR mixed state diverges 
in manner similar to the memory pattern in the sparse limit with l/|/log/|, but that the 
storage capacities of the other types of the mixed states do not diverge. In order to clarify 
this reason, we evaluated the overlaps of s types of the mixed states with the memory 
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pattern. The overlap of the OR mixed state converged to 1 in the sparse limit, while the 
overlaps of all of the other s — 1 types of the mixed states converged to 0. Moreover, the 
evaluation of the variance of the cross-talk noise on the naive S/N analysis provided the 
above qualitative causes. The threshold value of the OR mixed state corresponded to the 
threshold value of the memory pattern in the sparse limit. This was understood from an 
investigation of the overlap mentioned above. We conclude that the OR mixed state is the 
appropriate mixed state in the sparse encoding scheme. 

Finally, let us infer a relationship between the present theoretical results and recently 
reported physiological findings of Sugase et al. |T3| for the face-responsive cell in the IT 



cortex. Sugase et al. performed a single unit recording in the IT cortex of two macaque 
monkeys by showing them various visual stimuli, such as monkey- and human-faces with 
various facial expressions and simple geometric shapes. They measured the temporal change 
of the information carried by the firing of neurons for the classification of visual stimulus sets 
jnj. They found that the initial transient firing correlated well with rough categorizations 
(e.g. face vs. nonface stimuli), and the subsequent sustained firing represented more detailed 
information. Their results suggest that the neuron firing pattern is initially a superposition 
of patterns representing different faces or expressions, but it then converges to a single 
pattern representing a specific face or expression. To reinterpret this transient phenomenon 
by the "words" discussed in this paper, we might be able to say that the OR mixed state 
appears in the initial part of dynamics of neurons, and the network state finally converges on 
the memory pattern. To our regret, such transient phenomenon did not occur in this model. 
Recently, Toya et al. and Okada et al. have discussed an associative memory model with 



hierarchically correlated memory patterns [ITRIlGf. The hierarchically correlated patterns 



employed in Jl5| may capture the qualitative properties of the visual stimuli used by Sugase 
et al. Okada et al. reported the following transient phenomena that were similar to the 
results of Sugase et al.. The network state approached a mixed state at the first stage of the 
retrieval process. After that it diverged away from the mixed state and finally converged 
with the memory pattern that was closest to the input pattern. However, in their model, 
the mixed state was not the OR mixed state but the majority decision mixed state. It was 
thought to be the majority decision mixed state because the memory pattern was made to 
be 50% of the firing rate in |15|]. As mentioned before, the typical mixed state for the 50% 
firing rate is the majority decision mixed state. Considering our results and the results of 
[15| , we conjecture that the associative memory model stores the sparsely encoded memory 



patterns with a hierarchical structure, and the state of the model is not directly drawn into 
any memory pattern, but drawn into the memory patterns after being drawn once into the 
OR mixed state. This prediction agrees with the behavior of IT cortex neurons suggested 
by Sugase et al. 

APPENDIX A: 

The order parameter equation of the SCSNA is given as follows [p"2[ . 



Y = F ( ^2(rj tl - f)m^ + h + TY + az] , (Al) 



S 



m 

q 
u 

a 2 

r 



oo 



1 



Dz < Y l >, 



Dz z <Y >, 



(A2) 
(A3) 
(A4) 
(A5) 

(A6) 

Here < > stands for the average over the elements of retrieved memory pattern i]^ (1 < fi < 
s) that obeys Eq. @. Since the output function is F(-) = O(-), we apply the Maxwell rule 
fI^.[T0|j, and obtain the following expression, 



a J-oo 
aq 

aU 



U 



where, 



Dz 



dz z 2 



Y(r,,a,z) = e[ 'EW-fimp + h + ^ + az 



(A8) 



We also discuss the symmetric mixed state in this paper, where it is possible to put m M = 
m (1 < fi < s). By substituting these conditions into Eq. ( |AT| ) to (|A7|), we obtain the 
following Eq. (|A|) to ( |ATT|) , 

1 s-1 



m 



Es-iC v f(i-fy 



-v-l 



v=0 



erf( 



m(v - sf + 1) + h + ~ 



y/2. 



a 



— erf( 
Q 



m(v — sf) + h + | n 



v 7 ^ 



(A9) 



/ / v=0 



U 



xerf( 
1 



27TCT 



x exp 



m(t> - sf) +h + | 

£ s c,r(i-/r u 



(A10) 



u=0 



,m(v -sf) + h + S\2) 



V2a 



(All) 



Eq. 



decides the threshold value /i so that, 



/ = ^ + ^E.c«r(i-/) M 



xerf( 



m(t> — s/) + h + | 
9 



(A12) 



For the mixed state ~f( s > k \ f in the left term of Eq. (|A12|) only has to be changed into f( s < k \ 
Eq. (|7|) gives the mean firing rate f( s ' k \ The overlap M^ s ^ of an equilibrium state with the 
mixed state ^ s ' k ^> in Eq. (|9]) is shown as, 

M (s,k) _ ®( v - k) - / (s ' fc) _ 

^ 2/( s ' fc )(l - f(s,k)y vJ " j / 

xerf( "(* "«/)+* + § ), (Ai3) 
V 2a 
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FIGURES 

FIG. 1. 

Storage rate a dependency of overlap M^ 3 ' 1 ) on OR mixed state 7( 3,1 W s = 3 

FIG. 2. 

Storage capacity «c on OR mixed state -f^' 1 ^ 

FIG. 3. 

Asymptotic characteristic of storage capacity ac on OR mixed state 7 (s > 1 ). 

FIG. 4. 
Mean firing rate / dependency of optimum threshold value h c on OR mixed state 7( Si1 ) 

FIG. 5. 
Mean firing rate / dependency of storage capacity on majority decision mixed state y( s ' s / 2S ) 

FIG. 6. 
Mean firing rate / dependency of storage capacity on AND mixed state -f^ s ' s ^ 

FIG. 7. 
Mean firing rate / dependency of threshold value h c on majority decision mixed state 7( s,s / 2 ) 

FIG. 8. 
Mean firing rate / dependency of the threshold value h c on AND mixed state 7( s ' s ) 

FIG. 9. 
Mean firing rate / dependency of overlap m^' k ^ as recalled state is mixed state -y( 5 ' k ^ 
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Figure 1: T. Kimoto and M. Okada 
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Figure 2: T. Kimoto and M. Okada 



le+06 ,K memory pattern 

mixed-state s=3 
100000 -r % : \ mixed-state s=5 
mixed-state s=7 

10000 - 
1000 - 
100 - 
10 - 
1 - 



0.01 u 

le-08 le-06 0.0001 0.01 1 
Mean Firing Rate / 




Figure 3: T. Kimoto and M. Okada 
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Figure 4: T. Kimoto and M. Okada 
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Figure 5: T. Kimoto and M. Okada 
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Figure 6: T. Kimoto and M. Okada 




Figure 7: T. Kimoto and M. Okada 
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Figure 8: T. Kimoto and M. Okada 
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Figure 9: T. Kimoto and M. Okada 



